The present invention claims priority to priority document no. 2000-325223 filed in Japan on Oct. 25, 2000, and incorporated by reference herein.
1. Field of the Invention
The present invention relates to a user interface for executing an operation of object and a command input operation against a computer. In particular, the present invention relates to such user interface capable of instinctively and intelligibly executing an operation of object and a command input operation.
More particularly, the present invention relates to such a proposal pertaining to a user interface environment which instinctively and intelligibly expresses a set of operations by harmonizing a physical interface existing in a real space and a visual interface such as display for displaying a logical space of each computer. In particular, the present invention relates to such a proposal pertaining to a user interface environment capable of easily executing operation for linking mutually a plurality of apparatuses such as a data exchanging operation among these apparatuses.
2. Description of the Prior Art
Along with progress of technological innovation in recent years, a wide variety of general-purpose computer systems incorporating high value-added and highly sophisticated functions called work-stations or personal computers have thus been developed and commercially promoted in market with relatively small size and low price. Actually, these computer systems have widely been promoted in universities, research institutions, business enterprises, offices, and even in daily life of individual homes.
Generally, any of computer systems is driven in response to a command input by a user and provides an interactive processing environment by way of displaying the processed result on a display screen. Recently, it is noted that such a character-based user input environment called a CUI (Character User Interface) via a conventional keyboard represented by a DOS (Disk Operating System) shell screen has been shifted to a GUI (Graphical User Interface) that has realized a graphic-based user input operation. Under the GUI environment, such a desk-top with a simulated computer system and an innumerable number of icons are prepared on a display screen.
All of resource objects dealt in a computer system such as files are expressed as icons on a GUI-oriented desk top. By way of directly giving operation (such as click or drag and drop) to a display object on the screen to the icons individually symbolizing specific programs, data, folders, and devices, on the display screen via mouse operation, a user can instinctively operate a computer. In addition, a variety of functions such as menu bars and tool boxes, i.e., a variety of buttons for instantaneously activating every computer processing operation are prepared on the desk top. Accordingly, a style of the command input has become more intelligible for a user by intuition.
Owing to the introduction of the above mentioned GUI environment, a user is enabled to fully operate a computer without specifically learning titles of a specific command and how to use the command, and yet, without executing complex key input operations.
Typically, those coordinate-indicating devices such as a mouse, a track ball, a joy stick, a tablet, or a touch pad are cited as a user input devices operable under the GUI environment. Of these, the mouse has widely been propagated in the computer field, and accordingly, most of computer users are well accustomed to mouse operation based on the drag and drop operation. When newly introducing a computer to business offices or homes, it is not exaggeration to mention that there is no more need to specifically train users how to operate the mouse. As a matter of fact, the GUI based on the mouse operation has already been rooted among many users by way of providing them with a variety of general-purpose functions.
Further, in recent years, as an interactive input/output method in dealing with a computer and other data processing systems, study on interface via the real object has been propagated [refer to annotations 1, 2, 3, and 4].
According to such an interface utilizing the real object, unlike such a basic GUI operating method based on picture information via display output and a general-purpose input device such as a mouse, it is possible to extract such sophisticated function for dealing with real objects properly owned by the human being, and thus, there is such a possibility of utilizing the above interface for computer processing operation.
For example, it is possible for the above interface to quickly operate a plurality of real objects with both hands. Further, by virtue of real affordance [refer to annotation 5] from an real object, it is possible to constitute a more intelligible and intimate interface.
On the other hand, versatility and variableness provided by the GUI will be lost by the introduction of an real object as a means for implementing the user interface. It is relatively difficult to convert use of an real object designed for a specific object, i.e., any physical object, into another object such as computer processing operation. Most of such interface systems thus far proposed have been designed for specific uses, and actually, nothing have been designed to function as general-purpose platforms like the GUI.
Further, in recent years, there is a growing tendency in which interactive means for dealing with a computer or a data network has been shifted from PCs (Personal Computer) to information handling appliances. In other words, instead of executing all the works on a desk top of one computer, there is such a tendency in which users more frequently utilize a wide variety of electric apparatuses such as computer related electronic appliances and mobile apparatuses. Many of these apparatuses incorporate limited number of functions than that of general-purpose computers adopting the GUI, and thus, these apparatuses can utilize more simplified interface. Accordingly, it is conceived that such a simple interface capable of supporting linkage among a number of computers (data exchange appliances) at homes and offices should be designed as urgently as possible.
On the other hand, when computer processes are dispersed to a number of apparatuses due to propagation of network services, it will become greater importance for the operation to execute process for mutually linking a plurality of relevant apparatuses in order to exchange data between individual apparatuses.
For example, in order to transmit data contents comprising video image shown on a TV set in a home to a friend via E-mail using a portable telephone or in order to share a memorandum written on a white board together with an optional picture in an office or project these data on a projection screen, it is necessary to provide such an interface capable of readily executing an operation for exchanging data. However, inasmuch as it is quite difficult to predict combination of those applicable apparatuses and transmission in the form of data in advance, it is desired that such a system capable of easily executing proper function on the part of end users shall be formed.
It is an aspect of the present invention to provide such a useful user interface environment capable of instinctively and intelligently expressing data by way of harmonizing such a physical interface present in real space with a visual interface such as display for displaying logical space of computers.
It is another aspect of the present invention to provide such a simple user interface environment for supporting mutual linkage between a number of computers (data processing apparatuses) in homes and offices.
It is a further aspect of the present invention to provide such a useful user interface environment capable of easily executing operations for mutually linking a plurality of data processing apparatuses for exchanging data between individual apparatuses.
The present invention has been consummated by way of considering the above-referred problems. The first aspect of the present invention provides such a data input and output system comprising the following: a visual interface comprising a substantially flat operating surface for allowing installation of a physical interface and display function for displaying data on the operating surface; an identifying means for identifying the physical interface installed on the operating surface; and a data processing means for executing data processing operation in correspondence with identified result drawn by the identifying means and externally displaying processed result on the visual interface.
It should be understood that the term xe2x80x9csystemxe2x80x9d specified above refers to logical assemblage of a plurality of apparatuses (or such functional modules for realizing specific functions) without particularly questioning whether individual apparatuses and functional modules are installed within a single cubic casing or not.
Preferably, the above-referred physical interface comprises a substantially flat transparent structure, which comprises the following: a transparent area for allowing data displayed on the back-surface side of the transparent structure to be permeable onto the front-surface side; a property data displaying area which is disposed in part of the back surface or the front surface in order to display property data of the physical interface; and an identification data storing means which is buried in the transparent structure in order to sustain proper identification data.
In the practical form for implementing the present invention to be described later on, the above-referred physical interface is called a xe2x80x9ctilexe2x80x9d which is disposed on a flat operating surface.
The identification data storing means is formed in the name of xe2x80x9cwireless tagxe2x80x9d for example, which, in response to the reception of waves having a specific modulated frequency, returns AM-modulated waves corresponding to a proper identification data for example. In this case, an identifying means on the part of the data input and output system identifies proper data of the physical interface, i.e., the tile, based on so-called electro-magnetic transmission and reception method.
The inventive visual interface comprises a flat display such as a liquid crystal display for example. Preferably, the inventive visual interface comprises such a flat display surface integrated with a digitizer capable of reading pen input data via operational surface of the visual interface and the inventive physical interface disposed on the operational surface of the visual interface.
In this case, the above-referred data processing means executes a data processing operation in correspondence with combination of identification data held by the corresponding physical interface disposed on the operational surface and the pen input data added to the physical interface.
A plurality of physical interfaces may be disposed on the visual interface. Further, it is also possible for the data processing means to execute such a data processing operation in correspondence with combination of identification data held by two or more than two of the physical interfaces disposed on the operational surface of the visual interface.
Further, it is also possible for the data processing means to execute such a data processing operation in correspondence with combination of identification data held by two or more than two of the physical interfaces disposed on the operational surface of the visual interface and pen input data added to the physical interfaces.
Further, it is also possible for the data processing means to execute processing operations assigned to a specific identification data in response to the result of detecting the corresponding identification data from the corresponding physical interface disposed on the visual interface and then externally display the processed result onto the installed position of the corresponding physical interface in order that visual feedback can be effected.
Further, it is also possible for the data processing means to execute an application program assigned to a specific identification data in response to the result of detecting the corresponding identification data from the corresponding physical interface disposed on the operational surface of the visual interface and then externally display the result of executing the corresponding application program onto the installed position of the corresponding physical interface in order that visual feedback can be effected.
The inventive data input/output system may further comprise such a communication means for accessing external data resources. In this case, in response to the result of detecting a specific identifying data from any of the physical interfaces disposed on the operational surface of the visual interface, it is possible for the data processing means to access data resources assigned to the identification data and then externally display the acquired data onto the installed position of the corresponding physical interface in order that visual feedback can be effected.
Further, it is also possible for the data processing means to execute own linkage with real world physical object assigned to a specific identification data in response to the result of detecting the corresponding identification data from the corresponding physical interface disposed on the operational surface of the visual interface and then externally display data related to the corresponding physical object onto the installed position of the physical interface in order that visual feedback can be effected. For example, by way of feeding data from one of the physical interfaces onto another physical interface assigned to an real world physical object such as a printer, it is possible for the data processing means to output such a request against the printer for executing a printing operation.
Further, it is also possible for the data processing means to assign data held by one of the physical interfaces adjoining a specific physical interface to this specific physical interface in response to the result of detecting the corresponding identification data from one of the physical interfaces disposed on the operational surface of the visual interface and then externally display the assigned data onto the installed position of the corresponding physical interface in order that visual feedback can be effected. In this way, it is possible for the data processing means to transmit data received from other physical interfaces as though it acts as a container.
Further, it is also possible for the data processing means to assign data held by one of the physical interfaces adjoining a specific physical interface to remotely disposed physical interfaces related to this specific physical interface in response to the result of detecting the corresponding identification data from the corresponding physical interface disposed on the operational surface of the visual interface and then externally display the assigned data onto the position at which the remotely disposed physical interfaces are disposed. For example, by way of designating such a physical interface being the transmitter as the xe2x80x9cmasterxe2x80x9d and one of the physical interfaces being the transfer as the xe2x80x9cslavexe2x80x9d, when another physical interface feeds data to the master physical interface, it is possible for the master physical interface to instantaneously and instinctively transfer this data to the slave physical interface.
Further, it is also possible for the data processing means to assign data stored in one of the physical interfaces to another physical interface in response to the pen input operation executed across two or more than two of the physical interfaces. Further, it is also possible to externally display the assigned data to the position at which the other physical interface is disposed so that visual feedback can be effected.
Concretely, according to the data input/output system related to the present invention, by way of applying such a transparent tile which buries an identification data storing means like a wireless tag as an individual interface unit, it is possible to construct such an easy and intelligible interactive technique based on the arrangement of the tile on a flat display or tablet surface. Further, in response to the set-up of the tiles, i.e., the physical interfaces, the data input/output system reads identification data contained in the wireless tag, and then activates the corresponding processing operation. For example, the data input and output system accesses such data resources related to the tiles and then activates the corresponding function and application program. Further, based on physical positional relationship between plural tiles, it is also possible to combine basic capacity provided by individual tiles.
The second aspect of the present invention provides a method of inputting and outputting data for realizing an interactive technique via operation of the physical interfaces on a visual interface comprising a substantially flat operational surface for accommodating a number of physical interfaces and display function for displaying data on the operational display surface. The inventive data input/output method comprises the following: an identifying step for identifying physical interfaces disposed on the operational display surface; and a data processing step for executing data processing operation corresponding to the result of identifying the physical interfaces executed via the above identifying step and an ensuing step of externally displaying the processed result on the visual interface.
The data input/output method according to the second aspect of the present invention may further comprise a digitizing step for reading pen input data shown via the operational surface of the visual interface and the physical interfaces disposed on the operational surface thereof.
In this case, the above data processing step executes a data processing operation in correspondence with combination of identification data held by the physical interfaces disposed on the operational surface and the pen input data added to the physical interfaces.
Further, the above data processing step may also execute a data processing operation in correspondence with combination of identification data held by two or more than two of the physical interfaces disposed on the operational panel or the data processing step may also execute such a data processing operation in correspondence with identification data held by two or more than two of the physical interfaces disposed on the operational panel and the pen input data added to the physical interfaces.
Further, the above data processing step may also execute such a data processing operation assigned to a specific identification data in response to the result of detecting the corresponding identification data from the physical interfaces disposed on the operational surface of the visual interface and then externally display the processed result onto the position at which the corresponding physical interface is disposed in order that visual feedback can be effected. The process assigned to the identification data comprises such an operation to access an application program and data resources and linkage with an real world physical object for example.
Further, the above data processing step may also execute such a process to assign data held by one of the physical interfaces adjoining another physical interface in response to the result of detecting the corresponding identification data from the corresponding physical interface disposed on the operational surface of the visual interface and then externally display the assigned data to the position at which the corresponding physical interface is disposed so that visual feedback can be effected.
Further, in response to the result of detecting any specific identification data from one of the physical interfaces disposed on the operational surface of the visual interface, the data processing step assigns data held by one of the physical interfaces adjoining the above physical interface to a remotely disposed physical interface related to the above physical interface, and then externally displays the assigned data at the position at which the remotely disposed physical interface is present, whereby effecting visual feedback.
Further, the data processing step may also execute such an operation to assign data stored in one of the physical interfaces to another physical interface in response to the act of adding a pen input operation across two or more than two of the physical interfaces disposed side by side on the operational surface of the visual interface. Further, it is also possible to externally display the assigned data to the position at which the assignee physical interface is disposed, whereby effecting visual feedback as well.
The third aspect of the present invention provides such a memory medium which physically stores a computer software based on the computer-readable format. The computer software describes such a data input/output process to be compatible with execution on a computer system. The data input/output process provides such an interactive technique via operation of the physical interfaces on the visual interface incorporating a substantially flat operational surface for accommodating the physical interfaces thereon and such display function for displaying data on the operational surface. The inventive computer software comprises the following: an identifying step for identifying physical interfaces disposed on the operational surface of the visual interface; and a data processing step for processing data in correspondence with the result of identifying the physical interfaces identified via the above identifying step and an ensuing step of externally displaying the processed result on the visual interface.
The computer-readable memory medium according to the third aspect of the present invention comprises such a medium which provides a general-purpose computer system capable of executing a variety of program codes for example with a computer program based on a specific computer-readable format. The above-referred medium comprises such a detachable and portable memory medium such as a compact disc, a floppy disc, or a magneto-optical disc, or the like, for example. It is also technically possible to provide a specific computer system with a computer program via a transmission medium such as a network service based on wireless transmission or cable transmission.
The above program memory medium defines structural or functional cooperative relationship between the computer program and the memory medium in order to realize proper function of a specific computer program on a computer system. In other words, by way of installing a specific computer program into a computer system via the program memory medium related to the third aspect of the present invention, cooperative action is exerted on the computer system, whereby making it possible to achieve such an operating effect similar to that is attainable via the above-referred data input/output system and the data input/output method according to the first and second aspects of the present invention.
Still further aspects, features, and advantages of the present invention will more fully be clarified by the following description based on the description of the preferred embodiments and the accompanying drawings.